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Abstract 

In this exploratory quasi-experimental case study, we assessed the promise of a yearlong 
supplemental reading intervention with a small pilot group of at-risk first grade readers in an 
elementary school setting. Using standardized measures of reading proficiency, we found that after 
47 hours of one-on-one tutoring instruction, students read significantly more proficiently than did 
non-tutored students in a matched group of first grade peers in the same school. These results are 
encouraging in light of literacy research documenting the impact of one-on-one tutoring by 
qualified tutors of at-risk early grade readers. We used lessons learned from this pilot study to help 
inform and direct the necessary revisions and refinements of future reading interventions with the 
goal of building the school’s capacity to support the literacy development of at-risk readers so that 
they can catch up with their typically developing peers. 

Keywords: Response to intervention, early literacy instruction, one-on-one tutoring, at-risk 
readers 


Introduction 

Students arrive at school around 7:00 a.m. Parents drop them off by the cafeteria, where 
they get breakfast and meet their tutors. 

Denica waits for six-year old Keyonte each tutoring day (names are pseudonyms], 
Keyonte usually runs 10-15 minutes late. When she picks him up from the cafeteria, they 
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usually talk about his morning or the books he read the night before. These conversations 
help set the stage for the daily tutoring lessons. Denica intentionally asks open-ended 
questions to help engage Keyonte in purposeful and meaningful conversations. Most 
morning conversation starters are centered on breakfast or his clothes (he often has on 
matching sneakers and jackets, or new jeans with cool logos]. Gradually, their 
conversation becomes more centered on books and characters that are interesting to him. 

Once upstairs in the school library, they would settle in for their daily literacy lesson. 
Keyonte often works best when seated side by side at a table with Denica. Working on the 
floor proved to be too much of a distraction. Sitting beside him allows Denica to easily see 
how he is reading and point out key words and phrases during picture walks. 

Students like Keyonte are often regarded as "struggling" or "at-risk" readers. Most 
enter first grade with low literacy skills, and are therefore considered unprepared to fully 
engage in formal school-based literacy activities. On the other hand, these students come 
to school with an array of talents and home experiences that are critically important for 
their school success. In our study of what works for struggling first grade readers, we 
show how schools can bring about significant improvements in reading performance 
outcomes among these students, by catching them before they fall, providing them with 
intensive one-on-one instruction, and expert teaching will be necessary. For many of 
these students, as Marie Clay [2005] observed: "It is the individual adaptation made by the 
expert teacher to that child’s idiosyncratic competencies and history of past experiences 
that starts him on the upward climb to effective literacy performances." (p. 63], 

What research says about how to effectively reach and teach at-risk early grade readers 

When it comes to what works when teaching children to read in the early grades, we know 
one thing for certain: There are no silver bullets. On the other hand, a review of several 
decades of reading research and long-term data give us a better sense of what works — 
and, for the most part, it's what we're not doing, which, according to reading experts, 
includes early detection of reading difficulties, intensive instruction, and expert teaching 
for all children, especially for those who come from ethnically diverse and/or low-income 
families (e.g., Au, 2011, Clay, 2005; Taylor, 2007], 

For purposes of this article, we will focus on two main lines of literacy research and 
instruction. The first one examines the characteristics of effective reading instruction 
programs that have been shown to help at-risk readers catch up to their grade levels. The 
second line of research examines the degree to which these programs are similarly 
effective for all children, including those who come from ethnically diverse and 
economically disadvantaged backgrounds. 

What does it take to effectively reach and teach early grade at-risk readers? 

Literacy researchers and practitioners generally agree that it is possible to prevent 
reading problems for most children when they are provided with supplementary 
instructional support in the form of effective early and intensive literacy interventions 
(e.g., Clay, 2005; Snow, Burns, & Griffin, 1998; Wasik & Slavin, 1993], Some researchers 
have shown that almost all first grade children can learn to read, including those who 
enter school with low levels of literacy and who in the past would have failed to learn to 
read in first grade (e.g., Taylor, Critchley, Paulsen, MacDonald, & Miron, 2002], A recent 
U.S. Department of Education report concluded, after a review of evidence from available 
randomized controlled studies, that one-on-one tutoring by qualified tutors for at-risk 
readers in Grades 1-3 is effective (Institute of Education Sciences, 2003, p. iii). The report 
authors further noted "one-on-one tutoring of at-risk readers by a well-trained tutor 
yields an effect size of about 0.7. This means that the average tutored student reads more 


282 



Assessing the Promise of a Supplemental Reading Intervention / Mokhtari, Neel, Kaiser & Lee 


proficiently than approximately 75 percent of the untutored students in the control 
group." (p. 19). 

The above findings appear at odds with students' reading performance in many United 
States schools as revealed by state and national tests of reading proficiency. Report cards 
such as the National Assessment of Educational Progress (NAEP) indicate that since 2007, 
nearly two out of three 4 th grade students in the U.S. have had reading proficiencies below 
the level needed to do grade level work adequately (National Center for Educational 
Statistics [NCES], 2011). State assessments consistently show that as many as 30% of first 
grade students, on average, enter school with low levels of reading and writing. The 
number of students in need of reading assistance is much greater for students of low- 
income families, students with disabilities, and students representing culturally, 
linguistically, and racially different backgrounds. 

So why are there such high numbers of underachieving readers when the research 
evidence shows that reading problems are preventable for the majority of young children? 
The answer to this question is complex but depends, to a great extent, on what schools are 
doing or not doing to prevent and address students' reading difficulties. In our work with 
struggling readers in school settings, we often find that there is a gap between what is 
known about best practices in literacy instruction and what happens daily in practice, 
particularly in classrooms that have a high percentage of underachieving readers. The 
answer also depends on whether schools have the means and expertise to put in place 
systems for identifying children at-risk of reading difficulties, providing effective literacy 
instruction in the preschool and early grades, and supporting the professional learning 
and development of teachers. Consistent with these observations, literacy researchers 
have argued for several decades that "few students in the United States regularly receive 
the best reading instruction we know how to give" (Allington, 2011), and that classroom 
literacy instruction seldom reflects best practices as identified in the research (Taylor, 
Peterson, Pearson, & Rodriguez, 2010). 

Richard Allington, a leading literacy researcher who has spent many years studying 
exemplary elementary classroom teachers, has argued that as a literacy community, we 
know how to teach nearly every child to read by first grade. Unfortunately, few schools are 
doing what they need to do to help students most at-risk of reading failure. In an article 
published in Educational Leadership, Allington and Gabriel (2012) outline six elements of 
effective reading instruction that they assert "do not require much time or money—just 
educators' decision to put them in place" (p. 1). The key to reaching the goal of teaching 
every child to read by first grade depends on providing opportunities for every child to 
experience these research-based elements of reading instruction every day. According to 
Allington and Gabriel (2012), in order to help all students become competent, 
independent readers and writers, classroom teachers should (a) give students an 
opportunity to read something that appeals to their interests and needs, (b) read 
something they can accurately read and understand, (c) write something that is 
meaningful to them, (d) talk about what they read or write with someone, and (e) hear a 
fluent adult reader read aloud every day. 

Drawing from research on effective reading instruction during the last four decades, as 
well as her own research examining the "how" as well as the "what" of effective 
elementary reading instruction practices, Taylor and colleagues concur that many of the 
classroom literacy instruction practices she observed in thousands of classrooms over a 
period of several years are inconsistent with research-based instruction practices (Taylor, 
2007; Taylor et al., 2010). They argue that to reach the goal of helping all children in the 
elementary grades succeed in reading to their fullest potential, teachers and 
administrators within schools should make a concerted effort to work together to develop 
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and deliver a sound school-wide reading program. She further notes that while schools 
know that a wealth of information is available to help them move closer to helping every 
child become a reader, putting all the relevant pieces together remains a challenge. 
Moreover, she points out that ongoing professional development in which teachers work 
together within their buildings to reflect on their practices is an important first step in 
achieving this goal. 

Researchers, policy makers, and practitioners generally agree that the first three years 
of classroom instruction are critically important for preventing students from falling 
behind and preventing reading failure. During these critical years, schools lacking the 
expertise and/or the resources to put in place a system for providing expert reading 
instruction for all students are likely to create a pool of students who will become 
struggling readers. By and large, during the past several decades, schools have made 
substantial progress in addressing reading difficulties by designing effective early reading 
intervention programs. One of the most notable examples of successful early intervention 
programs is the Reading Recovery model, which uses one-to-one tutoring for struggling 
readers in grades 1-3 (Clay, 2005, Pinnell, Lyons, DeFord, Bryk, & Seltzer, 1994; Pinnell, 
Fried, & Estice, 1990], Another successful program is Success for All, which has a track 
record of providing successful school-wide tutoring interventions for students at-risk of 
reading difficulties (Slavin, Madden, Karweit, Livermon, & Dolan, 1990], 

Programs such as these, and others, use literacy practices that are supported by 
research evidence and that have been shown to work well for at-risk readers. Additionally, 
with new initiatives such as Response to Instruction (RTI], which is a part of the 2004 
reauthorization of the Individuals with Disabilities Act, research indicates that it is 
possible to substantially reduce the number of students classified as learning disabled. 
This legislation, according to Johnston [2011] and Allington (2010] enables schools to (a] 
provide increasingly intensive tiered instruction to help ensure that students having 
difficulty learning to read are provided with the requisite expert instruction, and (b] 
identify students who continue to have reading difficulties after receiving intensive 
reading instruction. The most commonly used form of RTI has three tiers of instruction 
ranging from conventional classroom reading instruction (Tier 1], to supplementary 
expert instruction delivered in small group settings (Tier 2], to targeted instruction 
provided in one-on-one tutorial settings for students most in need of reading assistance. 

Since the enactment of the IDEA legislation, there has been a great deal of interest 
within schools and districts to put in place tiers of instruction systems aimed at 
significantly reducing the number of students experiencing reading difficulties. To address 
the needs of students who are most at-risk of reading difficulties (i.e., those who are in the 
third tier of instruction], many schools have put in place various types of extended-day 
programs depending on their needs and resources. Allington (2012] describes four of the 
most commonly used extended school-day designs as follows: 

1] School-based remedial assistance with expert reading instruction. In this design, 
eligible students work with reading and/or special education teachers for an hour 
or more after school to accelerate literacy development. 

2] School-based tutoring with trained community volunteers, high school, or college 
students. Designs such as this often consist of only once or twice weekly sessions, 
although some do provide daily instructional support. 

3] School-based homework help/child care/recreation with paraprofessional or 
volunteer support. In this design, eligible students receive mostly homework 
assistance with corresponding recreational activities. 
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4] Community-based homework help/child care/recreation. There are actually fewer 
school-based than community-based after school programs currently operating. 
These programs, which are similar to school-based programs, are sponsored by 
organizations such as YMCA, Boys and Girls Clubs, churches, and other community 
groups (Allington, 2012, pp. 179-180], 

These designs vary in terms of target audience, staffing expertise, instructional focus, 
and intensity of instruction. Research on the effectiveness of these program designs is 
rather mixed. For instance, in one study, Wasik and Slavin (1993] found that programs 
using certified teachers resulted in significantly higher gains than programs using non- 
certified staff. However, other researchers (e.g., Davidson & Koppenhaver, 1993; Inverzini 
Rosemary, Juel, & Richards, 1997; Wasik, 1998] found that programs using non-certified 
personnel were as effective as those using certified teachers. The key to success in these 
programs, according to Allington [2012], appears to be related to "providing non-certified 
personnel with strong training, structured tutorials, and ongoing supervision" (p. 181]. 

Have reading programs been equally effective for nearly all at-risk readers? 

Looking beyond the reading programs that have been shown to be effective in helping at- 
risk readers catch up to their grade level, a growing number of literacy researchers and 
teacher educators have expressed concern that effective programs have not always been 
comparably successful for all at-risk readers, especially for those children who come from 
ethnically diverse and economically disadvantaged backgrounds (e.g., Au, 2011; Cochran- 
Smith, 2004; Compton-Lilly, 2007, 2009, 2011; Gorski, 2013; Ladson-Billings, 2009], 
These children, they argue, are often overrepresented among at-risk readers in schools. 
They bring a vast range of abilities, practices and life experiences that are often quite 
different from the practices and experiences they encounter at school. 

In light of these circumstances, one should not assume that a program or intervention 
that proves to be effectiveness for one group of students will be equally effective for other 
groups of readers. While serving as a Reading Recovery teacher trainer in a high-poverty 
school in Wisconsin, Compton-Lilly (2011] examined how well the program serves African 
American children, including children who did not complete the 20-week interventions or 
who were otherwise hindered by policies, which disadvantaged children who bring 
diverse life experiences to Reading Recovery classrooms. She found that there was a 20% 
difference in success rates between African American children and European American 
children when we considered all the children served in Reading Recovery. These authors 
argue that recognizing the funds of knowledge these children bring to school is an 
important first step to helping them become successful readers and writers. In his book, 
Reaching and Teaching Students in Poverty: Strategies for Erasing the Opportunity Gap, 
Gorski (2013] provides an insightful review of the research and instructional practices 
that hold promise for working with these children around literacy, an excellent analysis of 
why economic inequities exist and persist among public school students, and an overview 
of practical classroom-tested guidance for teachers and leaders who care enough to make 
a difference. 

The Present Study 

We collaborated with an elementary school on the design of a supplemental or tier 3 
before-school tutoring program, with the goal of enhancing the reading skills of a small 
pilot group of at-risk, first-grade students. Our proposed supplemental reading 
intervention program is fairly similar to the second program design described above in 
that it uses trained pre-service teachers and provides a highly structured and closely 
supervised reading intervention program. 
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The overall purpose of the study was to evaluate the effectiveness of this supplemental 
reading intervention on students' reading achievement outcomes. Specifically, we were 
interested in finding out whether participation in this supplementary tutoring program 
results in significant improvements in students' reading achievement as measured by 
scores obtained on nationally normed measure of reading proficiency. 

Instructional/Research Setting 

We conducted the pilot study in a local area elementary school located in a mid-size city in 
the southwestern United States. Opened in 2000, the target school is a Fine Arts magnet 
school operating within a large Independent School District (approximately 18,000 
students] in the southwestern United States. The school has 675 students in kindergarten 
through 5 th grade. Student population is 36% African American, 32% Hispanic, and 27% 
White with an Economically Disadvantaged rate of 65%. Forty-two full-time teachers and 
20 support staff serve these students. The school has five first grade classrooms with an 
average of 24 students per classroom. Of the 120 or so students in first grade, about 30- 
40% were designated as needing assistance in reading. 

Study Participants 

Participants in this case study consisted of 12 first grade at-risk readers who entered first 
grade with low levels of reading and writing skills. Students were selected for 
participation in the study based on teacher recommendations and student performance on 
district benchmark assessments data using the Texas Primary Reading Inventory test 
battery, which placed them in the lowest performing quartile among all first grade 
students in the school. Of the 12 students identified for tutoring, 5 were Hispanic (4 males, 
1 female] and 7 were African American (6 males, 1 female]. Although none of the students 
were identified as having a reading or learning disability, one student repeated first grade, 
one was identified as having attention deficit hyperactivity, and one was an English 
learner. Additional information about student demographics is provided in Table 1. In the 
section below, we provide a brief profile of each of the student participants. 

J.A. is seven years old. He likes to draw and enjoys spending time with his family. His 
mother works at a day care, and he spends a lot of time with his Granny. He enjoys 
mysteries and scary stories with monsters, and he likes playing math games at school. In 
his tutoring sessions, it is often difficult for him to focus and he gets off-task easily if he is 
not actively engaged in an activity. Initially, he had low confidence in his reading ability, 
but has visibly gained confidence through this program. 

Six-year-old V.A. enjoys playing at the playground. She lives at home with her mom, 
dad, and sisters. V.A prefers short, funny books, particularly about animals. At home, she 
does not have any books in her room but her sisters read chapter books so they have a few 
books around. During her tutorial sessions, V.A struggles to read fluently. She began 
tutoring with low confidence in reading and was very shy, but quickly gained confidence 
through her reading and enjoys choosing the book she would like to read during tutoring 
sessions. 

L.B. is six years old. She spends a great deal of time drawing, playing games on the 
computer, and socializing with her siblings. She currently lives with her grandmother and 
aunt and is the youngest of her many siblings. L.B says she enjoys reading, particularly 
animal books, picture books, and funny stories. At school, she enjoys playing games, 
especially math games. At home, her grandmother occasionally reads to her before 
bedtime. During tutoring sessions, L.B struggles to stay engaged. Her confidence level in 
reading is high during familiar reading, and is hesitant to attempt reading through more 
difficult texts. 
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E.C. is seven years. He loves playing soccer. At school, E.C enjoys learning, playing with his 
best friend at recess, learning math, and participating in science activities. He says he likes 
to read about mysteries, and books about super heroes, animals, sports, and cars. At home, 
he reads mostly with his Granny but there aren't any books at his house. E.C struggles 
mainly with phonemic awareness skills such as blending sounds. Overall, E.C has a desire 
to learn and finds reading enjoyable. 

Seven-year-old N.C. enjoys playing with his friends, and playing games on the 
computer. He claims that he loves to read, no matter the genre, and likes to write stories. 
At home, there aren't many books to read except in his Mimi's room. During his tutoring 
sessions, it is evident that he struggles to read words and sentences at age-appropriate 
levels fluently, which poses problems for him when trying to understand what he reads. 

H. C. is seven years old. His favorite activities include going to waterparks, watching T.V. 
and playing video games. H.C lives with his mom but visits his dad (non-English speaking] 
on the weekend. He also has an older brother who is at the same elementary school. He 
likes to be able to read books that are funny. H.C says that his mother sometimes reads to 
him at home on occasion. In tutoring sessions, H.C requires glasses to read, which he often 
forgets, and he struggles with word decoding and reading fluency. His confidence in his 
reading is low. 

J.H. is seven years old. He likes playing on the computer and with video games. His 
favorite character is the Hulk. J.H prefers reading non-fiction and books about superheroes 
and he reads with his mom in the evening with the few books he has at home. He gets 
easily distracted during tutoring sessions and finds it hard to focus on what he is reading. 

Six-year-old M.J. enjoys football, Spiderman, and playing with his younger siblings. He 
lives with his mom, who is an accountant, and his dad, a policeman, and has an older 
brother in 3 rd grade and a newborn brother. In his tutoring sessions, M.J. has trouble 
making connections between letter sounds in words longer than five letters and struggles 
with comprehension when reading grade level materials. 

J.M. is eight years old. He likes watching T.V. and drawing, and has interests in bugs and 
animals. He lives with his mom and has older sisters who are also in elementary school. 
J.M. rides the bus to school and is sometimes late to school because of it. He likes his tutor 
and looks forward to coming to school as a result. 

I. O. is seven years old. He likes playing on the playground, tag, and soccer. He lives at 
home with his parents and two brothers. 1.0. prefers to read books about spiders, soccer, 
animals, cars, and super heroes. At home, he reads with his parents and brothers and has 
several Dr. Seuss books in the house. 1.0. often arrives late to his tutoring sessions, which 
reduces the amount of time his tutor gets to spend with him. 

C.R. is six years old who enjoys drawing pictures and learning about fish. At home, he 
reads mostly with his mother. While they do not have books in his house, his mother 
frequently reads magazines and his grandmother reads newspapers. C.R. prefers to read 
books about sports and super heroes. He has excellent attendance at school and likes 
getting help in reading. 

E.S. is a six-year-old boy who struggled with reading but has made great progress 
throughout the year. He enjoys engaging in literacy conversations and reading books at 
home with his siblings, parents, aunts, uncles, and grandparents. He reports that his living 
room only has two or three books, although he has access to magazines at home and he 
like computer-reading games. He also likes math and science because he thinks they are 
cool and with reading, the pictures in the books let him know what he is reading. During 
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tutoring sessions, he especially likes to read about sports, superheroes, animals, car and 
trucks, book in series, and funny stories. 

Instructional Framework 

For purposes of this study, we developed a research-based instructional framework to 
help us organize and manage instruction. Our literacy instruction framework incorporates 
established instructional design characteristics in terms of content, organization, and 
management of literacy instruction. It also takes into account student needs as well as the 
needs of our tutors. For instance, we wanted to design a framework that incorporates the 
literacy needs of our target group of students with respect to early language and literacy 
skills. In addition, we developed a flexible the framework to enable our tutors to adapt 
lessons that are relatively easy to implement and evaluate, and to enables us to closely 
supervise and monitor the degree to which components of the framework are 
implemented as intended to the extent possible. 

Our instructional framework includes three key components including (a] a 20-minute 
reading or re-reading of an easy or familiar text, [b] a 20-minute shared or interactive 
reading of a self-selected text, and [c] a 20-minute interactive writing segment targeting 
specific writing skills and strategies. Together, the three components of the framework 
take approximately 50-55 minutes to complete. Appendix A provides a descriptive outline 
of the key components of the framework with sample activities. 

Consistent with the recommendations delineated in the newly developed Publishers' 
Criteria for the Common Core State Standards in English Language Arts and Literacy, 
Grades K-2 (Coleman & Pimentel, 2012], all curriculum materials used when tutoring our 
target students have three important characteristics: first, students have access to a mix of 
literary and informational texts that vary in terms of topics, length, complexity, and 
genres. We wanted to help ensure that students have opportunities to read texts that are 
rich and accessible on their own and with others in order to help build their knowledge, 
experience, and enjoyment of reading. Second, we made a concerted effort to select 
materials that appeal to students' interests and needs. We wanted to provide students 
access to interesting reading materials that motivate them to read so that they can do so 
independently in and outside of school. Third, we provided a sufficient number of leveled 
reading materials during each tutoring session so as to enable students to read texts they 
can read on their own as well as texts that are more complex that they may be able to read 
with tutor guidance. Tutors introduce higher-level texts and present them through read- 
alouds, shared readings, and other tutor-assisted strategies. 

Instructional Delivery 

Pre-service teachers pursuing degrees at a local university received training as literacy 
tutors and delivered the tutoring lessons three times a week for a period of 10 weeks 
during the regular fall and spring semesters of one school year. Tutor training took place 
as a part of a semester-long reading assessment and instruction course that pre-service 
teacher candidates complete in partial fulfillment of the requirements of a Bachelor's 
degree in elementary education. This course is designed to help pre-service teachers 
strengthen expertise in identifying students' reading difficulties and designing instruction 
aimed at helping students improve their early grade reading skills. The course includes a 
supervised field experience, which requires teacher candidates to apply what they learn 
about reading assessment and instruction in a local elementary school setting. 

Tutor Training 

Our approach to preparing effective tutors is guided by the ecological context of our at- 
risk participating readers. We challenge our tutors to think critically and reflectively about 
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productive ways of supporting the reading and writing development of these students, 
many of whom come to school with low levels of literacy. We want our tutors to think 
carefully about why they do what they do before rather than after thinking about what 
they do and how they do it. When tutors explore the underlying reasons why a particular 
child entered first grade with lower than expected reading skills, for instance not knowing 
all the letters of the alphabet, they are more likely to be constructively responsive to the 
needs of that child. They are also likely to become more thoughtful and reflective about 
their teaching. 

We planned and implemented our tutor training in three closely integrated steps. 
These steps include an initial training phase during the first four weeks of the semester, a 
30-minute debriefing phase, which takes place immediately following each tutoring 
session, and an individual consultation phase, which takes place throughout the semester 
depending on the needs of individual tutors. 

During the first four weeks of the semester, tutors receive instruction in approximately 
1.5 hours twice a week for four weeks. During this time, tutors also learn about the school 
setting in which tutoring takes place, meet with the school principal and teachers, and 
learn about the students they will be assigned to tutor. 

During the tutor training sessions, instruction and coaching typically consists of close 
reading and discussion of evidence-based literacy practices, using assessment data to 
inform instruction, and organizing instruction in tutorial settings using a common 
instructional framework. Supporting materials for the first component include a mix of 
readings such as "What at-risk readers need" (Allington, 2011),"FAD: Filtering, analyzing, 
and diagnosing reading difficulties" (Mokhtari, Niederhauser, Beschorner, & Edwards, 
2011], and "What's a tutor to do? (Roller, 2006], During the debriefing sessions, tutors 
engage in a guided tutoring lesson study and reflection with a focus on what worked well, 
and what needs improvements, as well as sharing of ideas and resources. Individual 
consultation is initiated either by the instructor or by the student depending on perceived 
needs. Tutors keep a reflective journal, which often serves as a source for identifying 
challenges, questions, or issues for discussion. 

Research Design 

Doing research in real-world school settings is complex. Our study takes place in a school 
setting with ecological constraints, which makes it not feasible or ethical to conduct 
randomized controlled experiments, the 'gold standard’ for establishing what works. For 
purposes of our study, because our target student participants were identified as at-risk 
readers with a unique set of needs and backgrounds, we were not interested in just finding 
out whether our intervention generally works for these students as a group, for doing so 
does little to tell us under what circumstances it does or does not work. More importantly, 
our goal was to use the data obtained from the intervention to better understand the 
conditions under which individual children, not just groups, succeed or fail to learn to 
read. 

We collected student and tutor assessment data at various times during the year to 
help us determine the promise of the literacy tutoring intervention for producing the 
outcomes it is intended to produce. We collected pre- and post- assessment data for all 
first grade students in the school. Doing so enabled us to make comparisons in reading 
achievement outcomes across three groups of students: At-risk readers who needed and 
received reading assistance (At-Risk Tutored], at-risk readers who needed but did not 
receive supplemental reading instruction (At-Risk Non-Tutored], and typically developing 
readers who did not need nor receive supplemental reading assistance (Typically 
Developing Peers]. Because we combined two approaches in our research design, (we 
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collected data multiple times and examine student performance across groups], our design 
can best be described as a quasi-independent group design with one equivalent group and 
one non-equivalent group design. 

When selecting our non-tutored comparison group, we used propensity score 
matching, a quasi-experimental technique, recommended by Guo & Fraser (2010] to find 
students similar to the intervention students in terms of their background characteristics, 
using information from school databases such as gender, ethnicity, school-administered 
reading benchmark test scores, participation in free or reduced lunch, and other 
demographic characteristics. We also randomly selected a group of typically developing 
readers from the rest of first grade students who did not need assistance in reading. 

Data Collection 

We assessed students' reading achievement outcomes at the beginning and end of the 
school year using the Gates-MacGinitie Reading test (MacGinitie & MacGinitie, 2004], This 
test is a standardized nationally norm-referenced, general reading ability test, which 
assesses foundational literacy skills including vocabulary and reading comprehension 
skills. It is commonly used test of reading ability with adequate technical adequacy as 
measured by reported reliability (reliability coefficients range from .89 to .93 for 
vocabulary and .87 to .94 for comprehension] and validity data (high correlations 
reported for studies correlating Gates reading tests with other tests of reading 
comprehension]. 

In order to determine the extent to which tutors adhered to the key elements of the 
instructional framework, we collected fidelity of implementation data in two ways. First, 
tutors completed a fidelity of lesson implementation protocol for each of the lessons they 
delivered (See Appendix A], Fidelity of lesson implementation consisted of checking off 
lesson components completed as-is, modified, or not completed. Second, two members of 
our team observed each tutor for at least one lesson on two occasions during the school 
year using a lesson observation protocol (See Appendix B], These data helped us monitor 
and assess the functioning of the tutoring lessons in action, collect implementation fidelity 
data, and make the necessary adjustments in the design of the intervention so as to 
enhance its overall functioning. We reviewed 100% of the lesson implementation fidelity 
forms completed by tutors and sought reliability for coding the observation protocols 
made by two members of our team for each tutor observed. An examination of the tutor 
fidelity of lesson implementation indicated that our tutors implemented lessons as 
intended or with slight adaptations about 85% percent of the time. Inter-rater reliability 
between two lesson observers was 89%. 

Data Analysis 

We collected student and tutor assessment data throughout the year to help us determine 
the promise of the literacy tutoring intervention for producing the outcomes it is intended 
to produce. In an attempt to determine whether the reading intervention had an impact on 
at-risk tutored students, we analyzed the data obtained in three different yet 
complementary ways. First, we examined the amount of tutoring time each student 
received over the course of the school year, and we analyzed the progress in reading 
proficiency achieved by students individually and as a group from fall to spring taking into 
account the total tutoring time invested. 

Second, we examined the progress achieved by tutored students in comparison to a 
matched group of at-risk non-tutored peers and in comparison to all typically developing 
first grade peers in the same school. We analyzed the data obtained by using Analysis of 
Covariance (ANCOVA], which enabled us to control for students' initial level of reading 
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ability using students' Gates-MacGinitie Fall semester scores as a covariate, and provided 
us with a cleaner measure of effect of our instructional intervention. 

Finally, with a relatively small number of students in our study, and in light of the fact 
that students' reading development changes over time, but not necessarily in the same 
way or at the same rate, we reviewed each student's growth individually, and examined 
student performance in relation to national proficiency standards so as to determine 
whether the student is on track to reach proficiency, or remains at-risk of reading failure. 

Results 

The results obtained are presented in three complementary ways: As the amount of 
tutoring time each student received over the course of the school year, the progress in 
reading proficiency achieved by students individually and as a group from fall to spring 
taking into account the total tutoring time, the progress achieved by tutored students in 
comparison to a matched group of at-risk non-tutored peers and typically developing first 
grade peers in the same school. Please note that our data analyses are based on a total of 
10 participating at-risk students since one of the students moved in the middle of the year, 
and one student completed the program but did not participate in all assessments. These 
results are presented in Tables 1-2 and Figures 1. Table 1 presents group demographics as 
well as the amount of time spent by the at-risk first grade students who received tutoring 
the fall and spring semesters of the school year. 

Table 1. Student Demographic Profiles 

At-Risk Tutored At-Risk Non-Tutored Typical Peers 

(n= 12] (n= 11) (n=10) 


Mean Age (SD) 

Gender 

Male 

Female 

Ethnicity 

African-American 
Caucasian 
Hispanic 
Special Need 

English Learner 
Special Needs 


6.67 (.65) 6. 

8 7 

4 4 

7 6 

0 1 

5 4 

1 1 

1 1 


(.64) 6.5 (.53) 

6 

4 

2 

6 

3 

0 

0 


How much one-on-one instructional time did each at-risk student receive over the course of 
the school year? 

As Table 1 shows, the ten first graders received an average of 17.85 hours of instructional 
time during the fall semester and 28.95 hours of tutoring during the spring semester for a 
total average of 47.33 hours (SD=4.58) of tutoring or roughly 2840 minutes. Individual 
tutoring time ranged from a low of 42 hours (2520 minutes) to a high of 53.95 (3195 
minutes). The average tutoring time our students received falls within the recommended 
44-80 hours range of instruction needed to substantially reduce the incidence of reading 
failure in a school system by accelerating at-risk students' reading proficiency to average 
levels of performance (Allington, 2012; Clay, 2005). 

What proportion of tutored students made sufficient progress in reading proficiency after 47 
hours of one-on-one reading instruction? 

An examination of the students individual reading progress data show that 9 out of 10 of 
the at-risk tutored students achieved significantly higher extended scale scores on the 
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Gates-MacGinitie test in the spring 2013 when compared to their performance on the 
same test in the fall 2012. These data are fairly consistent with literacy tutoring research 
suggesting that when tutored by a well-trained tutor, the average at-risk reader should be 
expected to read more proficiently than approximately 75 percent of the untutored 
students in the control group (Institute of Education Sciences, 2003], 

Are at-risk tutored first graders learning beginning reading skills at about the growth rate 
one would expect? 

At the end of first grade, the main concern is whether each student has developed 
adequate beginning reading skills to get a good start in reading. To determine whether a 
student has made good gain during grade 1, assessment experts recommend using 
National Curve Equivalents [NCEs], which are normalized standard scores with a mean of 
50 and a standard deviation of 21.06. NCEs measure progress in reading by describing a 
student's position within the norming group at successive times during the year or grade 
levels. As a general rule of thumb, experts agree that a student who maintains about the 
same NCE from fall to spring or earns a total score on a test level less than 7 NCEs has not 
changed relative to the achievement of students in the norming group. A student with an 
NCE score of 50 is roughly at grade level. Table 2 displays the average NCE scores of 
students in our three groups. An examination of these data indicates that at-risk students 
made gains of nearly 20 NCEs between fall and spring semesters while the scores of at-risk 
non-tutored students actually declined by nearly two points from 30.1 to 28.6. Typically 
developing first grade peers gained nearly 18 points from fall to spring. While it is evident 
that the NCE growth scores of the at-risk group of students were lower than typically 
developing peers, the reading progress made provide evidence that the reading 
intervention has made a significant difference in the reading skills of at-risk tutors. 

Table 2. Comparison of Reading Growth Rate of At-Risk Tutored, At-Risk No-Tutored, and 
Typically Developing Non-Tutored First Grade Peers 


Measures 

Fall 

M (SD] 

Winter 

M (SD] 

Spring 

M (SD] 

Gates-MacGinitie Reading Test (NCE]* 
At-Risk Tutored Peers 

21.75 (14.0] 

37.9 (14.36] 

40.3 (18.7] 

At-Risk Non-Tutored Peers 

30.1 (15.7] 

30.8 (19.8] 

28.6 (19.3] 

Typically Developing Peers 

60.0 (26.6] 

78.4 (22.19] 

77.67 (21.2] 

* NCE= National Curve Equivalent 





How proficiently did at-risk tutored students read when compared to at-risk non-tutored 
peers and to typically developing first grade students in the same school? 


Table 3 presents the means and standard deviations pertaining to the reading progress 
made by the three groups of students at three points in time during the school year (i.e., 
Fall, Winter, & Spring], Table 3 also includes the means that were adjusted for the effects 
of the covariate. Following Field [2009], we used these means rather than the original 
means to more accurately interpret the group differences reflected in our ANCOVA 
analysis. 
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Table 3. Comparison of Reading Proficiency of At-Risk Tutored, At-Risk Non-Tutored, and 
Typically Developing Non-Tutored First Grade Peers 


Measures 

Fall 

M (SD) 

Winter 

M (SD) 

Spring 

M (SD) 


Gates-MacGinitie Reading Test CESS)* 



Original Means 

Adjusted Means** 

At-Risk Tutored Peers 
(12.6) 

288.3 (22.6) 

343.0 (24.9) 

371.7 (43.5) 

393.83 

At-Risk Non-Tutored Peers 
(11.1) 

306.4 (28.2) 

329.1 (35.5) 

342.3 (43.9) 

351.09 

Typically Developing Peers 
(14.8) 

367.3 (56.7) 

433.1 (54.8) 

456.6 (50.3) 

421.19 


Notes: *ESS=Extended Scale Score; ** Means Adjusted For The Effect Of The Covariate 


Using a test of Between-Subject effects, we found an overall significant group effect of 
our reading intervention after controlling for initial group differences in reading ability, 
F(2,26) = 8.002, p = .002], An examination of the adjusted means (and contrasts) for the 
three groups shows that the at-risk tutored students had higher adjusted means after 47 
hours of one-on-one reading instruction (M= 393.83; SD= 12.6) than did their at-risk 
matched peers who did not receive any tutoring (M= 351.09; SD= 11.1). However, as Table 
2 shows, the adjusted means of the typically developing peers (M= 421.19; SD= 14.8) was 
higher than both the at-risk tutored non-tutored. We verified these results using the Sidak 
Corrected post hoc pairwise comparisons among the three group means and found that 
the at-risk tutored students outperformed the at-risk non-tutored students, (p = .04). The 
post hoc tests further showed that the typically developing students outperformed the at- 
risk non-tutored students (p = .001) but not the at-risk tutored students (p = .218) 
although they had a higher adjusted mean score. Figures 1 provide a visual depiction of 
these differences. 



Figure 1. Comparison of Reading Growth of At-Risk Tutored, At-Risk Non-Tutored, & 
Typically Developing Non-Tutored Peers 

Discussion 

The findings of this study are encouraging. Our pilot quasi experiment generally shows 
that it is indeed possible to significantly advance the early grade reading achievement 
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outcomes of at-risk first grade students when we provide them with a sufficient amount of 
individualized instruction by well-trained tutors. We analyzed the results using a suite of 
procedures to determine whether our pilot intervention achieved its intended outcomes, 
and functioned well in a local area school setting. 

Taken together, our findings revealed several promising outcomes. First, we found that 
9 of 10 of the at-risk tutored students achieved significantly higher extended scale scores 
on the Gates-MacGinitie test at the end of the intervention when compared to their 
performance on the same test before the intervention. These data are consistent with 
literacy tutoring research suggesting that when tutored by a well-trained tutor, the 
average at-risk reader should be expected to read more proficiently than approximately 
75 percent of the untutored students in the control group (Institute of Education Sciences, 
2003], 

Second, we found that students who received tutoring read more proficiently after 47 
hours of instruction when measured by Gates-MacGinitie, a standardized norm-referenced 
test of reading ability. These students read more proficiently at the end of the intervention 
than did non-tutored students in a matched group of first grade peers in the same school. 
However, when compared with the performance of typically developing readers, we found 
that these students received lower average reading proficiency scores than typically 
developing peers in the same school, although this difference was not significantly 
different. 

Third, we examined student performance to determine whether at-risk tutored first 
graders' beginning reading skills were at or about the growth rate one would expect. Using 
average NCE scores of students in our three groups of readers, we found that at-risk 
students made gains of nearly 20 NCEs between fall and spring semesters while the scores 
of at-risk non-tutored students actually declined by nearly two points from 30.1 to 28.6. 
Typically developing first grade peers gained nearly 18 points from fall to spring of the 
same year. While it is evident that the NCE growth scores of the at-risk group of students 
were lower than typically developing peers, the reading progress made provide evidence 
that the reading intervention has made a significant difference in the reading skills of at- 
risk tutors. 

When considering all aspects of this pilot study, we find that while these results are 
quite encouraging, especially in light of literacy research documenting the impact of one- 
on-one tutoring by qualified tutors of at-risk readers in grades 1-3, an achievement gap 
remains when comparing the reading proficiency of tutored students to that of their 
typically developing first grade peers who did not need extra assistance in reading. This is 
not too surprising since our students entered first grade with a significantly larger gap in 
literacy achievement than did typically developing peers. Closing this reading achievement 
gap will take additional instructional time in the form of one-on-one and/or small group 
instruction, which will help accelerate to average levels of performance the progress of 
children who show early signs of reading difficulty. Some of these children's reading 
progress typically falls within the lowest 20% of the enrollment in similar school settings. 

In light of these findings, it is worth noting that in order to help maintain the progress 
at-risk students made during the school year, opportunities need to be provided for them 
to read and write during the summer months. Research has shown that students in 
primary and elementary grades lose much of their reading ability when they do not read 
during the summer months when school is not in session. This reading loss has been 
shown to affect these students' reading performance when they return to school in the fall. 
Research has also shown that students from lower socio-economic backgrounds suffer 
greater summer reading loss than do students from upper socio-economic levels. The 


294 



Assessing the Promise of a Supplemental Reading Intervention / Mokhtari, Neel, Kaiser & Lee 


likelihood of summer reading loss is therefore more real for students who are poor and 
who have poorly developed language skills (Allington & McGill-Franzen, 2003], 

Taking a rearview mirror look at our yearlong journey, we learned valuable lessons 
from this pilot study that helped inform and direct the necessary revisions and 
refinements of the intervention with the goal of building the school's capacity to support 
the literacy development of at-risk readers so that they can catch up with their typically 
developing peers. We are eager to share these lessons with primary and elementary grade 
reading teachers and school leaders, who may be interested in putting in place reading 
intervention programs aimed at improving reading performance among underachieving 
students in the early grades. 

Lessons Learned 

In assessing what worked well for us in this pilot program, we attribute the improvement 
in reading proficiency among our target students to five closely inter-related contributors. 
First, we wanted to identify students who entered school with low literacy skills fairly 
early during the year and in first grade. Even though the school had underachieving 
readers in second through fifth grade, we wanted to design a reading intervention 
program that specifically targets students entering first grade. Investing in first grade 
reading development will is more likely to have an impact on reducing the incidence of 
reading failure in subsequent grades. 

Second, drawing from research and practices documenting what has worked 
particularly well in tutoring programs such as Reading Recovery, Success for All, and 
others, we wanted to provide these students with a sufficient dose of intensive instruction 
that is likely to result in improved reading achievement outcomes for these students. 
Although we know that the amount of instructional time needed to help close the 
achievement gap varies a great deal depending on student needs, we used the 
recommended margin of 44-80 hours of instruction as a general target (e.g., Allington, 
2012] in designing our reading intervention for these at-risk students. 

Third, we worked conscientiously to help ensure that our tutors, who were pre-service 
teacher candidates, were effectively prepared for their tutoring roles and responsibilities. 
As we indicated in the tutor training section above, our training focused on the challenges, 
issues, and questions that our target students were experiencing at that time. Tutors 
received intensive training during the first four weeks of the semester on evidence-based 
literacy practices, using assessment data to inform instruction, and organizing instruction 
in tutorial settings using a common instructional framework. This training was followed 
by daily debriefing sessions, which took place immediately following each tutoring 
session, and individual consultation, which took place throughout the semester depending 
on the needs of individual tutors. This model of tutor preparation was experiential and 
focused primarily on the school context in which tutoring took place. 

Finally, we attribute the impact of this pilot reading intervention to our school- 
university collaborative relationship, which proved essential in terms of identifying 
students in need of reading assistance, access to pertinent assessment data, providing an 
environment conducive to tutoring sessions, and enabling excellent communication 
among tutors, parents, and the first grade teachers of these students. 
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APPENDIX A 

LITERACY LESSON FRAMEWORK 


Lesson Component 

Description 

Time Frame 

Reading Familiar Text 

Session begins by having the child re-read a 
familiar text with the goal of building reading 
fluency and boosting self-confidence (Clay, 
1995; Pinnell, Fried, & Estice, 1990). Tutor 
conducts a running record as the child reads. 

15 minutes 

2.5 Minute Break 

Interactive Reading 

Shared book reading and writing is interactive 
experience whereby the child participates in 
guided reading and writing activities, thus 
allowing them to learn about how language 
works and to see themselves as readers and 
writers (Snow, Griffin, & Snow, 1998; Holdaway, 
1979). 

20 minutes 

2.5 Minute Break 

Interactive Writing 

Session concludes with the tutor modeling reading 
of (and writing about) a challenging new text. 
This is an opportunity for the child to read and 
write about texts that are rich and accessible in 
terms of content so as to help build their 
knowledge, experience, and enjoyment of 
reading and writing (Trelease, 2006). 

20 minutes 
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APPENDIX B 

TUTOR SELF-RATING OF LESSON IMPLEMENTATION FIDELITY PROTOCOL 


Please take a few minutes following each lesson to share insights regarding lesson implementation. 


Tutor:_ Start Time: _a.m. p.m. 

Tutee:_ End Time: _a.m. p.m. 



Implemented 

Comments 

Familiar 

Reading 

(15 Minutes) 

^As is^ 

© 


2.5 Minute Break 

Interactive 

Reading 

(20 Minutes) 

^*As is^ 

© 


2.5 Minute Break 

Interactive 

Writing 

(20 Minutes) 




Open-Ended Comments: 


1. Please describe aspects of this lesson that worked particularly well. 

2. Please describe aspects of this lesson that did not work well. 

3. Please describe what you will do next to address the aspects of the lesson that did 
not work as planned. 
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APPENDIX C 

LESSON FIDELITY OF IMPLEMENTATION OBSERVATION PROTOCOL 
Observer:_ 

Tutor:_ Start Time: _a.m. p.m. 

Tutee:_ End Time: _a.m. p.m. 



Implemented 

Comments 

Familiar 

Reading 

(15 Minutes] 

© 

© 


2.5 Minute Break 

Interactive 

Reading 

(20 Minutes] 

(&7s^ 

© 


2.5 Minute Break 

Interactive 

Writing 

(20 Minutes] 

CiZ) 



Overall Lesson Quality: 

3= Outstanding 
2= Fair 


Overall Lesson Pacing: 

3= Too fast 
2- About right 


1= Needs improvement 


1= Too slow 


Open-Ended Comments: 

1. Describe aspects of the lesson observed that are particularly strong. 

2. Describe aspects of the lesson observed in need of improvement. 

3. Describe recommendations for improvement. 
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